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Multiple Measures Assessments 


ow should colleges determine whether students are placed into 


developmental or college-level courses? 


Each year, colleges place millions of students into developmen- 
tal math and English courses upon enrollment. To do so, col- 
leges most often use a high-stakes placement test, which numerous research 
studies have shown to be highly inaccurate in determining how well students 
are prepared for college. As a result, these tests “underplace” many students 
into developmental education classes who would have been successful imme- 


diately if they had taken college credit-bearing courses instead. 


Developmental education courses are designed to give students the skills they need for 
success in college-level courses, but they also delay students’ enrollment in credit-bearing 
coursework, lengthen the time it takes them to earn a degree, and may decrease the chances 
they will ever graduate. Colleges could boost incoming students’ college-level course pass 
rates by improving the assessment tools they use to place those students, with the goal of 
minimizing underplacement and increasing the number of students taking college-level 
courses. Using more than one measure to assess students’ skills—a strategy known as a 
multiple measures assessment (MMA)—can be an excellent way to achieve this goal. 


MULTIPLE MEASURES ASSESSMENTS: 
AN ALTERNATIVE TO HIGH-STAKES TESTING 


Recognizing the need to increase the number of students enrolling in college-level courses, MDRC 
and the Community College Research Center (CCRC) have been studying alternatives to high-stakes 
standardized placement tests. 


Colleges have many choices of measures and typically select them based on their predictive validity, 
availability, ease of use, and cost. There is strong empirical evidence that the high school grade point 
average (GPA) is one of the best available predictors of student success in college. Unlike a one-time 
assessment, the GPA is an aggregate measure of performance over multiple years and reflects not 
only students’ content knowledge, but also the types of behavior that influence success in college, 
such as attendance and participation. Box 1 presents various options for determining course place- 
ment using multiple measures of college readiness. 


MDERC and CCRC collaborated to evaluate the impact of two MMA models—one based on place- 
ment formulas at seven 2-year state colleges in New York and another based on decision bands at 
four 2-year state colleges in Minnesota. The research team conducted a randomized controlled trial 
at the study colleges in each state, assigning incoming students in the study sample to either a control 
group, whose members were placed using traditional placement testing, or a program group, whose 
members were placed using MMA. 


The study found that students who are placed into college-level courses using MMA are more likely 
to complete gatekeeper courses—basic introductory or prerequisite college-level courses—than their 
counterparts who are placed into developmental courses using placement tests. These findings held 
for placements in math and English courses in the first semester and after three semesters. The study 
also found that student success rates can improve when MMA is applied and students who would 
have otherwise been placed into developmental education courses are instead referred to college-level 
courses. Other key findings from the study include the following: 


e Inthe first semester, students who were “bumped up” using MMA—that is, placed into a col- 
lege-level course when they would have otherwise been referred to developmental education using 
a traditional placement method—were more likely to enroll in and complete this college-level 
course than similar students in the control group. 


e By the end of the third semester, students who were bumped up to college-level courses using 
MMA were more likely to have completed their gatekeeper courses than their counterparts in the 
control group. 


e However, students who were “bumped down” using MMA—that is, placed into a developmental 


education course when they would have otherwise been referred to a college-level one—experi- 
enced a negative effect similar in magnitude to the positive effect experienced by students who 
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BOX 1 
COURSE PLACEMENT SYSTEM OPTIONS 


Some of these options for determining course placement in college can be combined with one another. 
For example, placement formulas and directed self-placement can be used together. 


Exemptions or waivers place students directly into college-level courses without placement testing 
if their score on a specific test or other measure exceeds a certain threshold. 


Decision rules place students using a sequence of rules or measures that compare students’ scores 
on each selected measure against a threshold in a predetermined order. If a student meets the thresh- 
old, a placement is generated; if not, another measure is applied. For instance, if a student’s math 
ACCUPLACER score is too low for college-level studies, the advisor advances to the next measure, 
the high school GPA. If the student does not meet the high school GPA threshold for college-level 
math, the advisor moves on to the next measure, until there are none left. If the student does not meet 
the threshold on any of the measures, the student remains in developmental math. 


Decision bands divide students into three groups: ready for college-level courses, needs develop- 
mental courses, and a middle range. Students in the middle range are further evaluated using addi- 
tional measures. For example, for students whose placement test scores fall just below the threshold 
required to enroll in a college-level course, advisors might review their high school GPA or their results 
on another measure before making a placement decision. 


Placement formulas use an algorithm based on an analysis of historical student data to weigh var- 
ious measures (for instance, SAT score, high school GPA, and highest-level math course taken) and 
generate a recommended placement. 


Directed self-placement allows a student to decide which course is suitable after a conversation 
with an advisor about test results, prior courses, and grades. A college can use this method on its own 
or in conjunction with any of the above methods. When it is used with another method, the advisor 
informs the student of the generated placement but gives the student the option to override it along 
with information about the pros and cons of doing so. 


were bumped up. Although a useful predictor of college course success, the MMA model based on 
placement formulas bumped down some students who would have benefited from taking col- 
lege-level courses as much as those who were bumped up into these courses. Students who were 
bumped up using MMA were roughly 9 percentage points more likely to pass a college-level math 
or English course within three semesters than they would have otherwise been if a traditional 
placement method had been used. In contrast, the students who were bumped down using MMA 
were roughly 8 percentage points less likely to pass a college-level math or English course within 
three semesters. This finding indicates that an acceptable score on one placement measure should 
outweigh subpar scores on other measures in order to improve students’ chances of success in 
college-level courses. 
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FOR WHOM DO MULTIPLE MEASURES ASSESSMENTS WORK? 


It is important not only to understand how these assessments work in the aggregate but to ensure 
that they benefit student populations of interest, especially as states look for viable solutions to close 
achievement gaps. The study conducted at the state colleges in New York tracked impacts (causal 
effects of the MMA model) on student placement in college-level courses, enrollment, and college 
completion across several demographic characteristics, including student’s race, gender, age, and Pell 
Grant recipient status (a proxy for family income). 


Students in all the racial or ethnic, gender, and Pell Grant recipient status subpopulations consid- 
ered in the study were placed into college-level courses at higher rates when using MMA (with the 
exception of male students in math courses). The positive effects associated with using MMA to place 
students in math courses were observed in all racial subgroups in the first semester. However, those 
impacts did not persist after three semesters. The placement formulas approach piloted by the state 
colleges in New York reduced the achievement gap in English. That said, policymakers need not con- 
sider only this approach to address inequities in student success. Findings from the evaluation of the 
decision bands MMA model piloted at the state colleges in Minnesota and on whether it was more 
effective will be available in late 2021. 


THE BOTTOM LINE 


Placement systems based on simple decision rules that take into account high school GPA, placement 
tests, and other available measures can provide students with more than one way to demonstrate 
college readiness. While the cost to set up an MMA system can be substantial, once in place, the 
expense to operate it is comparable to that of administering placement tests alone. Using an MMA 
system can help place more incoming students in college-level math and English courses. More of 
those students will in turn pass their gatekeeper courses and make significant progress toward their 
postsecondary education goals. 


For more information, see MDRC’s 2018 guide on implementing an MMA system for states and post- 
secondary institutions based on lessons learned from a pilot study. 
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